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3.2.4 Variance 


Consider two random variables X and Y with the following PMFs. 


Px(x) 


0.5 for x = —100 

0.5 for x = 100 (3.3) 

0 otherwise 


Py(v) 


1 for y = 0 

0 otherwise 


(3.4) 


Note that EX = Ei = 0. Although both random variables have the same mean value, their distribution is 
completely different. Y is always equal to its mean of 0, while X is either 100 or — L00, quite far from its mean 
value. The variance is a measure of how spread out the distribution of a random variable is. Here, the variance of Y 
is quite small since its distribution is concentrated at a single value, while the variance of X will be larger since its 
distribution is more spread out. 


The variance of a random variable X , with mean EX = y , x , is defined as 

Var {X) = E[{X-yi X ) 2 ]. 


By definition, the variance of X is the average value of (X — /Ay) 2 - Since (X — /Ay) 2 P 0, the variance is 
always larger than or equal to zero. A large value of the variance means that ( X — fi x ) 2 is often large, so X 
often takes values far from its mean. This means that the distribution is very spread out. On the other hand, a low 
variance means that the distribution is concentrated around its average. 

Note that if we did not square the difference between X and its mean, the result would be 0. That is 

E[X — yx] = EX — E[nx\ = Vx ~ Vx = 0. 

X is sometimes below its average and sometimes above its average. Thus, X — fj, x is sometimes negative and 
sometimes positive, but on average it is zero. 

To compute Var(X) = E [(V — /Ay) 2 ] > note f h at we nee d to find the expected value of 
g(X) = (X — /Ay) 2 , so we can use LOTUS. In particular, we can write 

Var (X) = E[(X - g x ) 2 } = J2 (*k ~ tix) 2 Px(x k ). 

xk&R.x 

For example, for X and Y defined in Equations 3.3 and 3.4, we have 

Var(X) = (-100 - 0) 2 (0.5) + (100 - 0) 2 (0.5) = 10,000 
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Var(y) = (0-0) 2 (l) = 0. 

As we expect, X has a very large variance while Var(Y^ = 0. 

Note that Var( JC) has a different unit than X. For example, if X is measured in meters then Var( X) is in 

meters 2 . To solve this issue, we define another measure, called the standard deviation, usually shown as <7 y, 
which is simply the square root of variance. 


The standard deviation of a random variable X is defined as 

SD(X) = = yVar(X). 


The standard deviation of X has the same unit as X . For X and Y defined in Equations 3.3 and 3.4, we have 

< 7 * = ^ 10 , 000 = 100 
&Y = v/o = 0. 


Here is a useful formula for computing the variance. 


Computational formula for the variance: 


Var(X) = E[X 2 } - [EX] 2 

(3.5) 


To prove it note that 

Var (X) =E[(X- Mx ) 2 ] 

= E[X 2 -2fx x X + f, 2 x ] 

= E [X 2 ] — 2 E [/ixX] + E \_Hx\ by linearity of expectation. 

Note that for a given random variable X, [Ax is just a constant real number. Thus, 

E[[l X X\ = [l X E[X\ = and E[f.J,' x ^ = , so we have 

Var(X) = E[X 2 ] - 2 /4 
= E[X>] -u? x . 

quation 3.5 is usually easier to work with compared to Var(X) = E (X — /lx) 2 ] • To use this equation, we 
can find E [X 2 ] = EX 2 using LOTUS 
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EX 2 

2 

and then subtract fl-^ to obtain the variance. 


^ ^ X k Px(xk), 

%k£Rx 


Example 3.19 

I roll a fair die and let X be the resulting number. Find EX , Var(X), and (Tx ■ 

• Solution 

o We have Rx = {1, 2, 3, 4, 5, 6} and Px{k) = for k = 1, 2, . . . , 6. 

111111 

.EX = 1- —b 2 ■ —1-3 - —b 4 • —h 5 - — + 6 - — 
6 6 6 6 6 6 

EX 2 = l- — +4-— + 9- — + 16-— + 25- — +36- 
6 6 6 6 6 


Thus, we have 

„ 7 . 

~ 2 ’ 

1 _ 91 

6 ~~ ~ 6 ~' 


Thus 


Var(X) = B[X 2 ] - ( EX) 2 = T 



91 

T 


49 

T 


« 2.92, 


cr x = yVar(X) « « 1.71 


Note that variance is not a linear operator. In particular, we have the following theorem. 


Theorem 3.3 

For a random variable X and real numbers a and b, 


Var (aX + b) = a 2 V ar(X) 

(3.6) 


Proof 

If Y = aX + b, EY = aEX + b. Thus, 

Var(y) = E[(Y - EY) 2 } 

= E[(aX + b-aEX-b) 2 ] 
= E[a 2 (X — /i X ) 2 ] 

= a 2 E[(X-fi x ) 2 ] 

= a 2 V ar(X) 
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From Equation 3.6, we conclude that, for standard deviation, SD (a X + fo) = | a | SD ( JC) . We mentioned that 
variance is NOT a linear operation. But there is a very important case, in which variance behaves like a linear 
operation and that is when we look at sum of independent random variables. 


Theorem 3.4 

If Ai, X r 2 j • • •, X n are independent random variables and X = X\ + X 2 + • • • + X n , then 

Var(X) = Var(Xi) + Var(X 2 ) + • • • + Var(X n ) (3.7) 


We will prove this theorem in Chapter 6, but for now we can look at an example to see how we can use it. 


Example 3.20 

IfX ~ Binomial(n,p ) findVar(X). 

• Solution 

o We know that we can write a Binomial(ri , p) random variable as the sum of Tl independent 
Bernoulli(p) random variables, i.e., X = X\ + X 2 + • • • + X n . Thus, we conclude 

Var(X) = Var(Xi) + Var(X 2 ) + • • • + Var(X n ). 

If Xi - Bemoulli(p) , then its variance is 

Var(Xj) = E[Xf] - {EX,) 2 = l 2 -p+ 0 2 • (1 - p) - p 2 = p(l -p). 

Thus, 

Var(X) = p{l - p) + p( 1 — p) -\ -b p{ 1 - p) 

= np(l — p). 


<— previous 
next —> 
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